Apache Spark is a flexible framework that allows processing of batch and real-time data. Its unified engine has made it quite popular for big data use cases. This book will help you to get started with Apache Spark 2.0 and write big data applications for a variety of use cases.It will also introduce you to Apache Spark – one of the most popular Big Data processing frameworks. Although this book is intended to help you get started with Apache Spark, but it also focuses on explaining the core concepts. This practical guide provides a quick start to the Spark 2.0 architecture and its components. It teaches you how to set up Spark on your local machine. As we move ahead, you will be introduced to resilient distributed datasets (RDDs) and DataFrame APIs, and their corresponding transformations and actions. Then, we move on to the life cycle of a Spark application and learn about the techniques used to debug slow-running applications. You will also go through Spark’s built-in modules for SQL, streaming, machine learning, and graph analysis.Finally, the book will lay out the best practices and optimization techniques that are key for writing efficient Spark applications. By the end of this book, you will have a sound fundamental understanding of the Apache Spark framework and you will be able to write and optimize Spark applications.

Apache Kafka is a great open source platform for handling your real-time data pipeline to ensure high-speed filtering and pattern matching on the fly. In this book, you will learn how to use Apache Kafka for efficient processing of distributed applications and will get familiar with solving everyday problems in fast data and processing pipelines.This book focuses on programming rather than the configuration management of Kafka clusters or DevOps. It starts off with the installation and setting up the development environment, before quickly moving on to performing fundamental messaging operations such as validation and enrichment.Here you will learn about message composition with pure Kafka API and Kafka Streams. You will look into the transformation of messages in different formats, such asext, binary, XML, JSON, and AVRO. Next, you will learn how to expose the schemas contained in Kafka with the Schema Registry. You will then learn how to work with all relevant connectors with Kafka Connect. While working with Kafka Streams, you will perform various interesting operations on streams, such as windowing, joins, and aggregations. Finally, through KSQL, you will learn how to retrieve, insert, modify, and delete data streams, and how to manipulate watermarks and windows.

Apache Superset is a modern, open source, enterprise-ready business intelligence (BI) web application. With the help of this book, you will see how Superset integrates with popular databases like Postgres, Google BigQuery, Snowflake, and MySQL. You will learn to create real time data visualizations and dashboards on modern web browsers for your organization using Superset.First, we look at the fundamentals of Superset, and then get it up and running. You'll go through the requisite installation, configuration, and deployment. Then, we will discuss different columnar data types, analytics, and the visualizations available. You'll also see the security tools available to the administrator to keep your data safe.You will learn how to visualize relationships as graphs instead of coordinates on plain orthogonal axes. This will help you when you upload your own entity relationship dataset and analyze the dataset in new, different ways. You will also see how to analyze geographical regions by working with location data.Finally, we cover a set of tutorials on dashboard designs frequently used by analysts, business intelligence professionals, and developers.

Apache Hadoop is a widely used distributed data platform. It enables large datasets to be efficiently processed instead of using one large computer to store and process the data. This book will get you started with the Hadoop ecosystem, and introduce you to the main technical topics, including MapReduce, YARN, and HDFS.The book begins with an overview of big data and Apache Hadoop. Then, you will set up a pseudo Hadoop development environment and a multi-node enterprise Hadoop cluster. You will see how the parallel programming paradigm, such as MapReduce, can solve many complex data processing problems.The book also covers the important aspects of the big data software development lifecycle, including quality assurance and control, performance, administration, and monitoring. You will then learn about the Hadoop ecosystem, and tools such as Kafka, Sqoop, Flume, Pig, Hive, and HBase. Finally, you will look at advanced topics, including real time streaming using Apache Storm, and data analytics using Apache Spark. By the end of the book, you will be well versed with different configurations of the Hadoop 3 cluster.

Jira is an issue tracker and project management system. With their latest release, the Jira team has now expanded their user base to agile teams as well as business teams. This book provides a comprehensive explanation covering all major components of Jira, including Jira Software, Jira Core, and Jira Service Desk.This book starts with an introduction to Jira's unique features and how it can be used as an issue-tracking tool. It will then teach you about how a new project is created by a Jira administrator, what responsibilities there are, and using correct and relevant schemes in your project. You will then learn how to configure project workflows and fields for project screens.You will understand the various permissions used in projects and the importance of project roles in Jira. Then, the book talks about the concepts of versions acting as milestones and using components when handling issues in your projects. It will then focus on analysing data using built-in reports and creating dashboards in Jira. At the end, it will discuss various best practices for users as well as project managers or project administrators.

Lua is a small, powerful and extendable scripting/programming language that can be used for learning to program, and writing games and applications, or as an embedded scripting language. There are many popular commercial projects that allow you to modify or extend them through Lua scripting, and this book will get you ready for that. This book is the easiest way to learn Lua. It introduces you to the basics of Lua and helps you to understand the problems it solves. You will work with the basic language features, the libraries Lua provides, and powerful topics such as object-oriented programming. Every aspect of programming in Lua, variables, data types, functions, tables, arrays and objects, is covered in sufficient detail for you to get started. You will also find out about Lua's module system and how to interface with the operating system.After reading this book, you will be ready to use Lua as a programming language to write code that can interface with the operating system, automate tasks, make playable games, and much more. This book is a solid starting point for those who want to learn Lua in order to move onto other technologies such as Love2D or Roblox.A quick start guide is a focused, shorter title that provides a faster paced introduction to a technology. It is designed for people who don't need all the details at this point in their learning curve. This presentation has been streamlined to concentrate on the things you really need to know.

Python's standard unittest module is based on the xUnit family of frameworks, which has its origins in Smalltalk and Java, and tends to be verbose to use and not easily extensible.The pytest framework on the other hand is very simple to get started, but powerful enough to cover complex testing integration scenarios, being considered by many the true Pythonic approach to testing in Python.In this book, you will learn how to get started right away and get the most out of pytest in your daily workflow, exploring powerful mechanisms and plugins to facilitate many common testing tasks. You will also see how to use pytest in existing unittest-based test suites and will learn some tricks to make the jump to a pytest-style test suite quickly and easily.

Blender is open source 3D creation software. With a long history and an enthusiastic community of users, it is the ideal choice for almost any kind of work with 3D modeling or animation. However, for new users, its power and flexibillity can sometimes be daunting, and that’s when you need this book!The book starts by showing you round the all-new Blender 2.8 user interface. You'll look at the most commonly-used options and tools, such as navigating in 3D and selecting objects. You will then use and manipulate one of the most important windows of the interface, the 3D View.You'll learn how to use essential tools for working with 3D modeling. To give your models the feel of real-world objects, you'll learn how to create materials and set up surfaces. You'll see how to use Physically-Based Rendering (PBR), which allows you to craft realistic surfaces such as wood, stone, and metal. You will also work with Eevee, a new real-time render engine in Blender. You will see how to add motion to objects, making use of Blender's impressive 3D animation features. Finally, you'll learn how to create scenes and organize them for rendering, and later add titles and effects using built-in Blender tools.By the end of the book, you will be able to use Blender 2.8 new UI, Create 3D Models with textures, Animations, and Render them in real-time using Eevee.

Blockchain is a technology that powers the development of decentralized applications.This technology allows the construction of a network with no single control that enables participants to make contributions to and receive benefits from the network directly.This book will give you a thorough overview of blockchain and explain how a blockchain works.You will begin by going through various blockchain consensus mechanisms and cryptographic hash functions. You will then learn the fundamentals of programming in Solidity – the defacto language for developing decentralize, applications in Ethereum. After that, you will set up an Ethereum development environment and develop, package, build, and test campaign-decentralized applications.The book also shows you how to set up Hyperledger composer tools, analyze business scenarios, design business models, and write a chain code. Finally, you will get a glimpse of how blockchain is actually used in different real-world domains. By the end of this guide, you will be comfortable working with basic blockchain frameworks, and develop secure, decentralized applications in a hassle-free manner.

Docker is an open source software platform that helps you with creating, deploying, and running your applications using containers. This book is your ideal introduction to Docker and containerization.You will learn how to set up a Docker development environment on a Linux, Mac, or Windows workstation, and learn your way around all the commands to run and manage your Docker images and containers.You will explore the Dockerfile and learn how to build your own enterprise-grade Docker images. Then you will learn about Docker networks, Docker swarm, and Docker volumes, and how to use these features with Docker stacks in order to define, deploy, and maintain highly-scalable, fault-tolerant multi-container applications.Finally, you will learn how to leverage Docker with Jenkins to automate the building of Docker images and the deployment of Docker containers. By the end of this book, you will be well prepared when it comes to using Docker for your next project.